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ABSTRACT 



This paper reports on a study that described the early 
stages of implementation of an external -reform model developed by a voluntary 
school -accrediting agency. The article centers on findings taken from 637 
schools that participated in the program called Outcomes Accreditation (OA) . 
The study examined the goals and expectations for improved student learning 
set by schools pursuing OA from 1987 through 1994. Three questions guided the 
research: What targets did schools select for improvement? Were the schools' 
expectations for improved learner outcomes guided by the accrediting agency's 
criteria for OA? and Do school characteristics and other factors make a 
difference in target identification or improvement expectations? The findings 
show that schools identified similar improvement targets from year to year. 
The targets that received attention in initial school -improvement efforts 
were constant, regardless of year, school characteristics, or state. For the 
schools participating in the study, a great deal of standardization already 
existed for what would first be improved in student learning. However, 
schools were flexible and amenable to policies that required them to change, 
though it took time to incorporate these policies into their intentions for 
improved student learning. The widespread acceptance of external guidelines 
moved the schools closer to having standardized expectations for improved 
quality with equity. (RJM) 
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The Goals Of School Improvement: A View From The Field 

by Anne K. Flanders 

The message is clear to those inside and outside schools; schooling must 
change so it results in better outcomes for all students. Even while there is general 
agreement about what must change, how improved student learning successfully 
can be accomplished is still the subject of debate. Furthermore, reliable processes 
that support improved student learning need to be identified. Reliance on good 
intentions and assumptions about how schools should improve is risky and 
resource consuming. 

A change in voluntary accrediting practices offered an opportunity to follow 
the implementation of an external reform model incorporating both school-level 
decision making regarding improvement and pressure to address commonly 
identified educational deficiencies. North Central Association (NCA) developed 
Outcomes Accreditation (OA) from research on school improvement, school 
effectiveness, and cognition and learning. The criteria for OA was formalized in 
1992 after seven years of field testing. Schools are required to address the 
accrediting agency’s loose guidelines for quality with equity and document 
improvements in student learning at the end of a three to five year cycle. However, 
this form of accreditation gives schools the responsibility for identifying what their 
own students needed to improve. 

Six hundred thirty-seven schools participating in OA from 1987 through 
1994 were examined. The contents of school improvement plans were analyzed to 
identify what was expected in these schools for improved learner outcomes. It was 
discovered that the schools selected similar learning targets regardless of year, 
school size, staff ratios, education level, governance, or state. In addition, over a 
period of three years, successive cohorts of schools gradually adapted their goal 



statements to include expectations of improved quality with equity that were 
required for accreditation. 

The Problem 

While there is no conclusive body of research confirming there is one best 
way of improving schools, some believe that schools cannot improve on their own. 
These persons feel schools must be directed in their efforts by outside agencies. 
School improvement policies set by these agencies can have the advantage of being 
equitable in their expectations for improving the academic achievement of all 
students. Those identified by individual schools have potential to enlarge already 
existing between-school differences in student learning. On the other hand, it is not 
clear that policy is the best lever for achieving improved learning outcomes. Many 
educators also believe that mandating school improvement without engaging the 
people in schools has unintended but negative effects on learning. 

More information about alternatives to school improvement initiatives that 
attempt to take advantage of the strengths of centralizing and decentralizing is 
needed. This study was conducted to describe in part the early stages of the 
implementation of an external reform model developed by a voluntary school 
accrediting agency. In it the similarities and differences in targets and expectations 
of schools seeking accreditation based on improved learner outcomes are identified. 
Three questions guided this research: 

• What targets did schools select for improvement? 

• Were the school’s expectations for improved learner outcomes 
guided by the accrediting agency’s criteria for OA? 

• Do school characteristics and other factors make a difference in 
target identification or improvement expectations? 




I,. 



Review of the Research 



The reasons for educational reform and school improvement are solid. 

They are needed. Smith 1995, Drucker 1993 believe that most people can not find, 
organize, communicate, and act on knowledge effectively enough to meet the 
demands of today’s workplace. In fact, Schlectly (1997) fixes the percentage of 
the population that currently meets high enough expectations to perform these tasks 
at 20%. As a result, they and others in business, policy, and education predict that 
productivity in the service sector and in technological development will suffer. 

It is not surprising that there is agreement in educational policy circles to 
improve the quality and the equitable distribution of learning (Ravitch 1995, Tyack 
and Cuban 1995, Smith 1995). Furthermore, educational improvement is within 
reach. What is known about academic achievement in other developed nations and 
what is being discovered about learning through research in cognitive science 
clearly indicates that American students can do better (Elmore 1996, Ravitch 1995). 

From this agreement, unprecedented activity has begun to set standards that 
will result in the improvement of schooling outcomes (Lewis 1997). However, the 
Department of Education (Lewis 1997) and the American Federation of Teachers 
(AFT 1997) report that progress in the development of educational standards has 
been incremental and varies by state. AFT’s (1997) critique of state-level efforts 
reveals several concerns about the quality of standards being set in 49 of 50 states. 
First is a fear that these standards will not address the “core areas” - mathematics, 
science, social sciences, and English. Second, that the standards identified will not 
be leveled or subject specific resulting in further nonstandardization of learning 
content. Finally, they will not control the 60-80% of curricular content AFT 
estimates is needed to ensure uniformity between schools. 
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Despite a strong showing of support for rigorous educational standards not 
all educators agree that they can improve student learning. Biddle (1997) thinks 
assumptions about the equalizing effects of standards actually divert policymakers’ 
attention from real problems that are associated with school funding and child 
poverty. In addition, Eisner ( 1995) predicts they will further distract people from 
paying attention to what really is needed in their schools. 

Furthermore, the assumption that these standards will improve school 
outcomes for students is further weakened by research on student achievement. 
Based on his analysis of student achievement, Jencks ( 1972) suggested over two 
decades ago that the largest share of academic achievement is attributable to 
students’ background factors, not schools. In support, others researchers Berliner 
and Biddle (1995), have connected students’ socioeconomic status to learning 
through the persistent association of lower academic achievement with the 
conditions of poverty. Faced with these results some educators believe that it is 
foolish to expect schools to take responsibility for educational outcomes that are the 
results of social and cultural influences (Berliner and Biddle 1995, Goodlad 1994). 
However, Goodlad ( 1994) and others also recognize the research on effective 
schooling demonstrating that schools can alter some of the selecting and sorting 
processes resulting in lower outcomes for disadvantaged students. 

Perhaps the real issue some educators have with common educational 
standards is inflated expectations for school reform or educational improvement. 
Research conducted on school improvement demonstrates that improved student 
achievement is complex. It requires more than a single standard about what is 
taught or to whom it is taught can accomplish. Elmore ( 1996) and McLaughlin 
(1987) are among those who have observed that educational improvement is the 
outcome of several interrelated factors. Foremost in their opinion are: a focus on 
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improving student learning, the use of networks of educators to support the 
change, mindfulness of the content and the process of changing, and the allowance 
of systemic adaptation to the pressure of change and schools’ reaction. 

Based on what is known about school improvement, it appears that 
professional educational organizations may be especially well suited to supporting 
school reform initiatives. Darling-Hammond and McLaughlin ( 1995) and Wagner 
(1993) are among those who have suggested that these organizations have the 
advantage over governmental policy bodies in supporting change in schools and 
accommodating school differences. Many of these orgaiuzations incorporate a 
focus on educational improvement, negotiate goals with their membership rather 
than impose them, and use networks in their membership to span the perspectives 
of different educational roles and organizations. Furthermore, most have begun 
educational improvement processes directed at improving student learning (Elmore 
1996, Wagner, 1993). 

However, the effectiveness of educational improvement efforts including 
professional agency collaboration needs further study to verify they actually 
contribute to improving learning Unfortunately, little has been published that might 
indicate what educators involved in these efforts intend to improve about student 
learning in their own schools. Therefore, an important iiutial step in gathering this 
information will be to identify what is selected for improvement under the loose 
criteria and miiumal direction supplied by professional orgaiuzations. 

Purpose of This Study 

This investigation is part of a larger project following the initiation and 
implementation of NCA OA. It was conducted to identify the targets and 
expectations for improved student learning set by schools pursuing OA from 1987 
through 1994. First, the contents of goals from 637 school improvement plans 
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were analyzed. Targets and expectations for learning improvement were identified 
for 2870 goals. Second, yearly, school governance, size, staffing ratios, level 
(elementary, junior high-middle school, high school), and state comparisons were 
made to find out if these factors were associated with similarities or differences in 
target selection or expectations for improving student learning. 

Subjects 

The subjects of this study were the 637 schools who chose to pursue NCA 
OA between 1987 and 1994. Criteria for schools’ sets of improvement goals were 
not formalized by NCA until 1992. Therefore, OA plans taken from schools 
participating in field testing prior to 1992 are treated a single group. Table 1 
provides a yearly breakdown of the number of schools submitting plans, the states 
in the NCA region represented, the average number of goals per plan, and the total 
number of goals examined for each year and field testing. 

Table 1 Yearly Breakdown Of OA Participation 





Number of 
Schools 


Different 

States 


Average 

Goals/Plan 


Goals 

Submitted 


Pre-1992 


68 


14 


4.3 


293 


1992 


165 


17 


4.7 


772 


1993 


320 


17 


4.7 


1496 


1994 


84 


1 


4.3 


360 


TOTAL 


637 


20* 


4.5 


2921 



* In all twenty states had schools seeking candidacy during this study, though several states 

participated some years and not others. 
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None of the schools were selected randomly to participate in this 
investigation. They became the subjects of this study either because they chose to 
participate in field testing or because they elected to pursue OA from 1992 through 
1994. The pre-1992 schools are the entire population of OA candidate schools 
receiving accreditation in 1993 and 1994. They began their improvement processes 
between 1987 and 1991. The 1992 and 1993 schools are the entire cohort seeking 
approval for initiating OA candidacy. The 1994 sets of improvement goals are a 
sample of those submitted that year. They include only the goals from a one state in 
the NCA region. 

The improvement plans come from schools that vary in size, location, staff 
to student ratios, education levels, and governance. Included in the sample are 
schools from 19 different states and one overseas Department of Defense region, 
with levels of schooling spanning pre-school through grade 12, and differing 
school sizes and staffing ratios, and public and private governance. Forty five 
percent of the schools were high schools, 23% middle or Junior high schools, and 
32 % elementary schools. The average enrollment in these schools was 710 
students, and staffing averaged 49 full-time professional staff members (FTE). 
When these figures are compared to those for US schools in 1993 (DES 1993) 
enrollments are 40% larger and FTEs 47% greater than national averages. 

However, the student-to-staff ratio of 14.6 is close to the 1993 average of 14.7, 
and the ratio of private-to-public schools (8-to-92) matched that of the states from 
which the schools came. 

Methods 

NCA does not tell schools what to improve when they choose OA. The 
improvement targets are the school’s own, selected on data about students’ current 
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performance. However, constraints in the form of criteria for school improvement 
plans are in place. They influence expectations for outcomes in the school’s goals. 

To meet OA guidelines a school submits a set of goals that forms a 
coordinated plan and focuses on student learning, distribution of expectations for 
improvement among all students, establishment of expectations for “complex 
behaviors” (see Appendix I), address of higher level thinking, and involvement of 
the school’s staff (NCA 1994). During 1992 and 1993 it was recommended that 
improvement plans include five goals; three addressing cognitive outcomes and two 
affective (NCA 1992). Three to five years are taken to complete improvements 
once plans have been endorsed by peer visitation to schools and peer reviewers at 
the NCA Annual Meeting in Chicago. 

Copies of school improvement plans submitted for peer review were 
collected at the annual meeting of the accrediting agency in 1992, 1993, and 1994. 
Characteristics of each goal contained in these plans were identified and coded. 
Exemplars of coding can be found in Appendix II. Exemplars were submitted to 
various groups of educators in the NCA region, and the coding was found to be 
reliable (Wick and Gose, 1994). 

The ten categories used for coding expected outcomes include seven 
complex behaviors that were identified by NCA (see Appendix I), in addition to 
expectations for the achievement of learning indicators, process implementation, or 
organizational outcomes unrelated to learning (Wick and Sarterfiel 1992). The 
specific curricular or extra-cocurricular area targeted for improvement in each goal 
was identified. The 19 target categories used were established through a survey of 
the goals submitted in 1992 and 1993 (Flanders 1993). Because NCA uses 
measurability, higher level skills, level of learning, focus on student learning, and 
equity as indicators of improved quality, these factors were used in the same way in 




coding. Therefore, the agreement or lack of agreement the goals had with criteria 
for quality was also coded. 

Descriptive and comparative analyses were made on the basis of several 
factors including year (pre- 1 992, 1992, 1993, and 1994), state, governance, 
school level, size, and staffing ratios. Descriptions and comparisons were used to 
address four questions. These questions were: 

• Do schools identify the same target areas for improvement and 
expectations for learning outcomes from year to year? 

• Did the content of goals change to adapt to OA criteria for quality 
with equity? 

• Which OA criteria seemed most easy to adapt to goal statements? 

• Do school characteristics make a difference in target selection or 
expectations for improvement? 



Results 

Do schools identify the same target areas for improvement and expectations for 
learning outcomes from year to year? 

OA candidates participating in field testing prior to 1992 through the 1994 
sample indicated their preferences for curricular improvement by selection of 
targets. The majority of goals examined targeted language arts, values related to 
learning, or tools and processes that support learning (see Table 2). Twenty-five 
percent of the goals submitted targeted language arts, another 28% targeted student 
values related to learning, and 25% targeted processes and tools that support 
learning. Of the remaining 22% of the goals, 1 1% addressed other content areas 
and 1 1% could not be linked to student learning. Similar preferences for target 
selection were found in the 1994 sample. The percent targeting language arts, 
values supporting learning, or tools and processes supporting learning in 1994 



varied no more than 3% from those in Table 2 for the pre-1992 through 1993 



schools. 



Table 2 Percent Of All Improvement Goals Targeting Specific Curricular 

And Co/Extra-Curricular Areas For Improvement From 1987 Through 1993 



I. Goals for subject areas 




36% 


Language arts 


25% 




Social sciences 


3% 




Mathematics 


3% 




Sciences 


2% 




Fine arts 


2% 




Health/physical education 


1% 




Foreign Languages 


0% 




Manual Arts 


0% 




II. Goals for student values related to learning 


28% 


Socialization/rule following 


16% 




Self-esteem 


8% 




Social contribution/service 


3% 




Autonomy/independence 


1% 




III. Goals for tools and processes related to learning 


25% 


Problem solving strategies 


11% 




Study skills 


11% 




Use of technology 


3% 




Creative thinking 


0% 





Expectations for improvements that were in compliance with OA criteria 
initially were low but increased over time (see Figure 1). Throughout the study 
schools also identified indicators of learning (improved testing outcomes or counts 
of student behaviors), implementation of learning processes, and organizational 
goals for improvement. Goals that were unclear or unrelated to expectations for 
student improvement virtually had disappeared by 1993. Prior to 1992, only 1 1% 
of all goals stated what students would do (complex behaviors) as a result of 
improvement efforts. In contrast, 75% of the 1994 sample’s goals had 
expectations for improved complex behaviors. The previous year, only 54% of all 
goals from the state that was used for the 1994 sample had expectations for 
complex behaviors. 
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Figure 1 All Goals By Type Of Outcome, Pre-1992 - 1994 
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Like targets, expectations for student learning focused on three complex 
behaviors (see Appendix 1). Thirty-six percent of all goal statements set 
expectations for students to improve the responsibility to "caring for self and 
others.” Another 23% looked for enhanced communication, and 21% anticipated 
improvements in “problem solving and creative thinking.” Complex behaviors 
seldom indicated for improvement were; making, fixing, growing things ( 1%); 
impact of sciences and technology (2%); global implications for the social sciences 
(4%) and making mathematical decisions using shape, pattern, and size (6%). 

Did the content of goals change to adapt to OA criteria for quality with equity? 

Prior to 1992, goal statements in school improvement plans exhibited both 
equity and quality as defined for OA about 12% of the time. The percent of goals 
addressing criteria set in OA guidelines improved by 1994 (see Figure 2). Equity 
was the last criteria to be integrated. By 1994 equity was evident in 84% of the 
goal statements from the sample state. The previous year only 54% of all goal 
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statements from this state had equitable expectations. The other criteria were also 
evident in most goals the last year of this study. Ninety-seven percent of all goals 
focused on student learning, 87% were selected through analysis of data on student 
performance, 71% integrated curricular areas (complex activities), and 76% percent 
targeted higher level skills. 



Figure 2 Percentage Of Goal Statements With Quality Indicators 
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Which OA criteria seemed most easy to use in goal statements? 

As can be seen in Figure 2, schools improved in the use of all the OA 
criteria with each passing year. However, some criteria was integrated more 
rapidly than others (see Table 3). Integrating skills and knowledge and targeting 
higher level skills as outcomes improved the most during the early stages of OA 
development and implementation. Focus on student learning 
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and measurability increased more than other criteria between 1992 and 1993. 
However, the greatest improvement overall was in targeting higher level skills and 
while the least was seen in equity. 



Table 3 Increase In Criteria Used In School Improvement Goals 



Criteria 


Pre-1992 to 1992 


1992 to 1993 


TOTAL 


Equity 


3% 


9% 


12% 


Student focus 


1% 


23% 


25% 


Data based 


5% 


20% 


25% 


Integration of curricular areas 26% 


1% 


27% 


Higher skills 


19% 


14% 


33% 



Do school characteristics make a difference in target selection or expectations for 
improvement? 

Target and expectation selections were compared on the basis of school 
characteristics including year, level, enrollment, staffing ratios, governance, and 
state. Correlations between percentages of target areas selected and outcomes 
indicated for improvement were strong (see Table 4) regardless of year, level, 
enrollment staffing ratios, and governance. However. Correlations between the 
percentage of each expectation identified in goals by state were weak. 

Comparisons were then made for significant differences by state in target 
selection and expectations. Based on these comparisons, no significant difference 
was found between states in percentage of target areas selected [F (19, 380) = 
0.067, p > .05]. However, when the percentage of outcomes indicated for each 
complex behavior, indicators of learning, implementation of learning processes. 
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and organizational outcomes were compared by state they were significant [F (19, 
200) = - 5.23 , p < .001]. Because some states submitted very few school plans 
(less than 10) during this study, they were eliminated and a second analysis of 
variance was conducted on the remaining 14 states. The findings were significant 
for differences in expectations between states [F ( 13, 140) = - 7.49 ,p < .001]. 
However, once again, no significant difference was found for target selections [F 
(13,247)= 0.039, p > .05]. 



Table 4 Correlations For Improvement 





Targets Selected 


Expectations Identified 




idf=9) 


idf=17) 


Levels 


Elementary /Jr. High 


0.90** 


0.91** 


Elementary/High School 


0.93** 


0.93** 


Jr. High/Middle School 


0.97** 


0.98** 


Enrollment quartiles “ 


first and second 


0.94** 


0.95** 


first and third 


0.94** 


0.95** 


first and fourth 


0.95** 


0.93** 


second and third 


0.96** 


0.95** 


second and fourth 


0.95** 


0.95** 


third and fourth 


0.95** 


0.95** 


Students/FTE quartiles ^ 


first and second 


0.94** 


0.97** 


first and third 


0.87** 


0.96** 


first and fourth 


0.80** 


0.91** 


second and third 


0.95** 


0.97** 


second and fourth 


0.94** 


0.94** 


third and fourth 


0.96** 


0.96** 


Governance 


Public and Nonpublic 


0.75* 


0.97** 



significant at the .01 level for a nondirectional test 
significant at the .001 level for a nondirectional test 



Mean enrollments for each quartile are: first, 413 students; second, 583 students; third, 
813 students; and fourth, 873 students. 

Mean students per professional staff member (FTE) for each quartile are: first, 10 
students; second, 12 students; third, 16 students; and fourth, 19 students. 
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Conclusions and Discussion 



Do schools identify the same target areas for improvement and expectations for 
learning outcomes from year to year? 

Schools identified similar improvement targets from year to year. The 
average school improvement plan had four or five goals targeting language arts , 
values related to learning, and processes supporting learning. Most of the values- 
oriented goals were focused on improving student behavior in schools. The 
processes supporting learning were split evenly between problem solving strategies 
and study skills (information collection and organization or inductive thinking). 
Often, plans with five or more goals had at least two goals for language arts 
(reading, writing, or combinations of reading, writing, speaking, and listening). 
About 40% of the plans also included a goal for another subject area (mathematics, 
science, social studies, or fine arts). 

The similarity of targets in school improvement plans may have been due in 
part to external pressures from superordinate agencies. In addition, schools also 
used similar sources for achievement data and took similar fist steps that may have 
been the most logical for initiation of student learning outcomes improvement under 
OA criteria. Regardless, the result was a healthy degree of standardization in 
school improvement plans rather than variation. 

Patterns of similar target-area selection by schools from the same state or 
same school district were strong. They could be seen in OA school candidacy 
materials and were verified in a survey of NCA state directors in 1993. As a result, 
of state or school district influence some targets always were selected though never 
others. School district policy also was used to Justify target selection. Often, 
schools from certain districts would reiterate the same goals, use the same 
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information as resources, or have the same resource person select and analyze their 
data for target selection. 

The educators participating in OA processes were required to use data-based 
decision making by accreditation guidelines. From their documentation, it was 
evident that they used similar resources to describe student achievement in the 
school and select targets. Virtually every school used standardized testing to justify 
their target selection. Surveys of staff, students, parents, and community were also 
popular sources of information about perceptions of student performance. In 
addition, in states where performance testing was mandated this testing was 
featured regularly in the school’s self-study description and justification of targets. 

Admittedly, the goals that schools identified for their first cycle of school 
improvement did not cover everything that policymakers or researchers have 
identified for improvement. The lack of goals for mathematics, sciences, social 
sciences, foreign languages, health and physical education, and the arts is 
worrisome. However, these subject areas may be the focus of later OA 
improvement cycles. 

For several reasons it is too early to speculate whether educators involved in 
OA are failing to address these areas. First, this study concentrated only on the 
initial cycle of schools’ improvement efforts and OA is meant to sustain continual 
improvement through successive cycles. It is unlikely that schools could repeat 
goals for things they have already improved because they would have difficulty 
documenting growth or change in student learning. Secondly, a single OA cycle 
probably is insufficient for covering everything that is in need of educational 
improvement. Finally, the subject areas that were not selected for improvement 
targets were not ignored by schools. They continued to be taught and learned as 
part of the regular mandated course work for students. However, they were not 
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singled out for additional efforts to improve their teaching and student learning 
outcomes in this cycle. 

Regardless, the goals schools select for improvement in the future will bear 
watching. If mathematics, sciences, and social sciences are not indicated for 
improvement in successive cycles of OA, intervention is needed. The inclusion of 
school-related values is also a question. Until 1994, NCA recommended OA 
school improvement plans contain two affective goals. No recommendation was 
made after that, so the number of goals stressing values that support learning may 
have declined in plans. 

It was clear that schools could indicate what would be improved but they 
did not identify what students would do as a result of their improvement efforts at 
the beginning of this study. However, by the end of this investigation expectations 
for improvement were becoming standardized among schools seeking OA 
candidacy. This will be discussed more thoroughly in the next section. 

Did the content of goals change to adapt to OA criteria for quality with equity? 

Expectations for the outcomes of student learning changed in school goal 
statements. The goals identified by schools beginning OA were increasingly more 
valuable as a basis for improvement. They demonstrated growing compliance with 
OA criteria for educational quality (use of higher level skills and knowledge) with 
equity each year. However, it took about three years for most goal statements in 
school improvement plans to reflect these guidelines. 

I feel it will take longer to implement these guidelines in school 
improvement than it did to include them in the goals. TTierefore, it will be 
important to verify the outcomes of school improvement efforts made by the OA 
candidates initiating their improvement cycles from 1992-1994. Obviously, proof 



that the effort was worth it and verification that a professional organization can be 
effective in mediating and directing change will come with these results. 

In addition, some of the criteria will probably be more difficult to implement 
in teaching, as some were harder to use in the simple activity of writing goal 
statements. However, there is evidence of organizational learning within the 
accrediting agency that should contribute to reducing some of the difficulty of 
subsequent school efforts. The capacity to address OA criteria was not lost from 
one cohort of schools to the next. Successive cohorts did not have to recreate their 
understanding of the criteria. Because of this, each cohort of OA schools was able 
to build on past work and improve on their predecessor cohort’s use of OA 
guidelines in improvement goals. This suggests that the accrediting agency’s 
members retained and shared knowledge about the application of criteria that 
encourage its evolution of use in practice. 



Which OA criteria seemed most easy to adapt to goal statements? 

A focus on student learning was the easiest criteria for schools to use; 
equity was the hardest. Because a focus on student learning is congruent with the 
work of schooling it may not have taken much of a leap to accommodate. I believe 
that the school organizations in this study OA had greater difficulty with the OA 
concept of equity because it requires of them a shift from common practice and 
accepted beliefs. Essentially their understanding of equity had to be expanded 
from one that accepted equal access to schools (even though different treatments 
and expectations were acceptable) to include common expectations for student 
outcomes. 

It seems apparent to me that equity was improved in these schools as a 
result of this shift. In addition, equity also improved between these schools 
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because participants standardized their expectations for learning improvement 
according to OA guidelines. However, I do not think that standardizing 
expectations for improvement within and between schools will be enough to 
overcome the more significant effects of socioeconomic disadvantages. Differences 
in resources for facilities, materials, and teacher qualifications are not addressed by 
these efforts, nor are the environmental stress factors experienced by children in 
poverty. Ultimately, governments will need to deal with most of the inequality 
between schools that affects learning outcomes because they have the base to 
change things that lie outside the school’s control. 

Do school characteristics make a difference in target selection or expectations for 
improvement? 

The school’s state made a difference in expectations for improvement. 
School characteristics of size, staffing, education level, and governance did not. 

The influence of the state was reflected through the rate of school adaptation to the 
criteria. Sometimes it was harder for schools to integrate state mandates with OA 
criteria. If the state set priorities for improvement, these were reflected in 
expectations or targets. When states had expectations for improved testing 
outcomes, indicators were commonly used by the state’s schools for goals. 

At other times schools indicated they selected OA criteria as a way to 
manage external pressures from mandated reforms. NCA state directors also 
verified this in a 1993 survey. About one-half said that schools in their states used 
OA as a process to deal with state rules and regulations for school improvement. 

A quarter of the directors also agreed that state mandates for school improvement 
interfered with school improvement processes under OA. 





Summary 

The targets that received attention in initial school improvement efforts that 
concentrated on learner outcomes were constant, regardless of year, school 
characteristics, or state. At least for the schools participating in this study, a great 
deal of standardization already existed for what should first be improved in student 
learning. However, schools also were flexible and amenable to policies that 
required them to change, though it took time to incorporate these policies into their 
intentions for improved student learning. 

In the beginning states made a difference in which criteria were used for 
expected learning outcomes though this difference was minimized over time. 
Regardless of the state, schools could take advantage of the practices of previous 
OA participants in the NCA region. The necessity for each cohort to relearn 
incorporation of criteria in goal statements or overcome obstacles arising from other 
policy bodies was reduced. As a result of the widespread acceptance of external 
guidelines moved these schools closer to having standardized expectations for 
improved quality with equity. 
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Appendix I 



Complex Behaviors Identified for Student Learning by NCA_ 



1. Technology’s impact and its basis in science (science, technology, and 
conservation) 

2. Making decisions using quantity, shape, location and pattern (math using 
numbers, patterns, and shapes) 

3. Problem solving and critical thinking (appropriate for all curricular areas) 

4. Communicating (reading, writing, speaking, listening, non-verbal, using 
fine arts). 

5. Caring for self and others (social or team work needs, goal setting, study 
skills or habits) 

6. Making, growing, and fixing things (a trade area using hands and equipment, 
hands on) 

7. Global implications of the social studies (social studies, citizenship, cultural 

awareness) 

NCA, A Focus on Student Success: A Handbook for Schools Seeking Outcomes 
Accreditation. 1992 
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Appendix II 



Improvement Target Areas And Exemplars Taken From OA 
School Improvement Plans 

GOAL CATEGORIZATION BY LEARNING TARGETS 

Following are examples of several goals classified by their primary 

academic target. 

• Language arts: 

“Students will demonstrate ability to comprehend the main and 
subordinate ideas in written work and select appropriate ways to 
communicate these ideas.” 

• Math: 

“Students will solve complex math problems involving several 
steps and operations.” 

• Problem-solving and critical thinking skills: 

“Students will improve their ability to solve, analyze, and evaluate 
problems using logical steps and appropriate resources.” 

• Study skills: 

“Students will assume responsibility for their learning 
through preparation, use of timelines, application of inquiry skills, 
cooperation with others in group tasks, and the development of 
personal standards for the quality of their work.” 

• Social sciences: 

“Children will share knowledge of their heritage, language, culture, 
and life experiences.” 



CODING FOR QUALITY AND EQUITY 
Expectations for Outcomes 

For the purpose of this study, a target-area goal was considered a student 
learning outcome (SLO) if it focused on skills or knowledge used by the students. 
Therefore, complex behaviors, indicators, and processes were all considered 
SLOs. Goals that were either unclear or focused on targets unrelated to student 
learning were not SLOs. Following are three examples of goals from school 
improvement plans that can be classified as student learning outcomes: 



• Complex behavior: 

“Students will increase their knowledge and ability to use various 
technology as tools for learning, information management, and 
communication.” 

• Indicator of learning: 

“Reading comprehension scores will improve.” 

• Implementation of a process supporting learning 

“Students will make portfolios in preparation for application to 
higher education or future employment.” 

Goals that do not address the use of knowledge and skills include 
indicators, processes, outcomes for parents, teachers, and organizations. Student 
learning goals for two different complex behavior strands follow. 

• Communication: 

“In all subjects students will demonstrate the ability to produce 
written documents in standard English (mechanics, usage, 
spelling)” 

• Caring for self and others: 

“Students will make responsible choices for their social and 
emotional health”. 

An indicator targets a narrow outcome for improvement through test scores, 
grades, or change in a count of student behaviors (absenteeism, library material 
checkout, discipline notices, counselor contacts, survey responses, teacher 
observations). Indicators do not provide in-depth information about how useful 
this is for students, but they usually imply which complex behavior the school 
wishes to improve. Two examples of indicators are: “Scores on weekly spelling 
tests will improve.” and “Students will be able to restate feelings and responses in 
non-judgmental terms.” 

Process goals are related to student learning, but they fall short of 
describing students’ use of complex behaviors. Like indicators, they are an 
important part of school improvement processes but are not precise targets for 
students’ use of knowledge and skills. Two examples of process goals are: 
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“Students will increase self esteem through goal setting/’ and “Girls will be 
encouraged to take part in higher level math and science courses.” 

Goals for school or organizational outcomes are related to improvement of 
the school but, unlike indicators and processes, are not directly connected to 
student learning. Therefore, it is even more difficult to tie them to the active use of 
skills and knowledge by students. Two examples of organizational outcomes are: 
“School employees will feel positive, empowered, and their accomplishments will 
be recognized.” and “Parents and community members will become significantly 
involved in the achievement of vision, goals, and objectives.” 

Goals that are unclear don't stand on their own as explanations for students’ 
use of knowledge and skills. Therefore, it is impossible to determine whether the 
school is attempting to improve something of value. Three examples of unclear 
goals that indicate target areas relating to complex behaviors follow. 

“Children will say positive things about each other.” 

“Students will demonstrate rational and irrational thinking.” 

“Art - all areas and all techniques.” 

Goal Categorization by the Level of Learning Being Addressed 

Each goal also was classified by the level of student learning it described. 
Three categorizations were used: integrated learning requiring active use of skills 
and knowledge, skill or concept acquisition requiring recall, and unclear. 

Examples of goals follow. 

• Integrated learning: 

“Students will apply their knowledge of historical, economic, 
political, and geographic patterns to analyze five themes in 
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American History: war, peace, expansion, depression, and 
globalization.” 

“Students will master the use of tools to construct and repair objects 
they use in everyday life.” 

“Students will develop the ability to analyze problems in various 
situations and curricular areas through the use of concrete 
mathematical models.” 



Skill or content acquisition: 

“Students will use “I” statements rather than engage in name 
calling.” 

“The writer will use proper punctuation on written assignments and 
tests in English.” 

“Students will locate and interpret information using 
developmentally appropriate resource materials.” 



Unclear 

“Students will understand the consequences of their actions.” 

“The student’s self-esteem will improve.” 

“ 80 % of the students will increase written communication skills 
80 % of the time by achieving at or above grade level norms on 
portfolio assignments.” 

“Students will learn to teach themselves.” 
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Goal Categorization for Measurability 



Goals that are clearly measurable in terms of student learning outcomes 
describe the active use of knowledge and skills that can be observed and recorded, 
perceived and reported, or documented by artifacts. The following are examples of 
clearly measurable goals written by OA candidacy applicants: 

“Students will use technological resources to improve oral and written 
communications in all academic areas.” 

“Students will demonstrate self-discipline to improve their social skills 
specifically when: a. interacting together , b. accepting responsibility, 
c. coping with criticism.” 

“Children will demonstrate the ability to work cooperatively in diverse 
groupings for various activities related to the school curriculum and its 
community.” 



If a goal cannot be authenticated by observable use of student knowledge or 
skills and the school supplied no data justifying its selection, it is considered 
immeasurable. The following four examples are of goals with measurement 
problems. 

“Students will reflect higher morals on the moral response surveys.” 

“Students will become more responsible by increasing attendance and 
reducing late homework assignments.” 

“Students will follow rules; Students will become successful later in life.” 
“Citizenship proficiency scores will increase to 85%. ” 



Goal Categorization for Equity 

Within-school equity requires that all students be given the same 

expectations for improved learning. Therefore, equitable goals address improved 

learning outcomes for all of the school’s students. The three goals that follow are 

equitable because they apply to every student in the school. 

“All students will demonstrate the ability to vary their writing style 
including vocabulary and sentence structure for different readers and 
different purposes.” 
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“Students will understand and demonstrate an acceptance of individual in 
social and academic situations.” 

“ Students will build the skills they need to become independent, life-long 
learners who are able use inquiry to acquire the knowledge they need.” 



Goals that specifically include or inadvertently exclude persons or groups of 
students have equity issues. If a target for a goal is not applicable to all of the 
school’s students then it is not considered equitable. Following are examples for 
each type of coding for inequitable expectations. 

• Academic achievement: 

“Ninth and tenth grade students will show an increase in the 

numbers of students passing classes.” 

• Socioeconomic status 

“Free lunch students will show an increase in achievement scores.” 

• Student behavior: 

“Student incidents of cheating will be reduced.” 
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